Rank/Norm Regularization with Closed-Form Solutions: Application to Subspace Clustering
نویسندگان
چکیده
When data is sampled from an unknown subspace, principal component analysis (PCA) provides an effective way to estimate the subspace and hence reduce the dimension of the data. At the heart of PCA is the EckartYoung-Mirsky theorem, which characterizes the best rank k approximation of a matrix. In this paper, we prove a generalization of the Eckart-Young-Mirsky theorem under all unitarily invariant norms. Using this result, we obtain closed-form solutions for a set of rank/norm regularized problems, and derive closed-form solutions for a general class of subspace clustering problems (where data is modelled by unions of unknown subspaces). From these results we obtain new theoretical insights and promising experimental results.
منابع مشابه
Sparse Subspace Clustering by Orthogonal Matching Pursuit
Subspace clustering methods based on `1, `2 or nuclear norm regularization have become very popular due to their simplicity, theoretical guarantees and empirical success. However, the choice of the regularizer can greatly impact both theory and practice. For instance, `1 regularization is guaranteed to give a subspace-preserving affinity (i.e., there are no connections between points from diffe...
متن کاملScalable Nuclear-norm Minimization by Subspace Pursuit Proximal Riemannian Gradient
Trace-norm regularization plays a vital role in many learning tasks, such as low-rank matrix recovery (MR), and low-rank representation (LRR). Solving this problem directly can be computationally expensive due to the unknown rank of variables or large-rank singular value decompositions (SVDs). To address this, we propose a proximal Riemannian gradient (PRG) scheme which can efficiently solve tr...
متن کاملNorm Regularization Algorithm for Image Deconvolution
Up to now, the non-convex l p (0 < p < 1) norm regularization function has shown good performance for sparse signal processing. Indeed, it benefits from a significantly heavier-tailed hyper-Laplacian model, which is desirable in the context of image gradient distributions. Both l 1/2 and l 2/3 regularization methods have been given analytic solutions and fast closed-form thresholding formulae i...
متن کاملروشهای تجزیه مقادیر منفرد منقطع و تیخونوف تعمیمیافته در پایدارسازی مسئله انتقال به سمت پائین
The methods applied to regularization of the ill-posed problems can be classified under “direct” and “indirect” methods. Practice has shown that the effects of different regularization techniques on an ill-posed problem are not the same, and as such each ill-posed problem requires its own investigation in order to identify its most suitable regularization method. In the geoid computations witho...
متن کاملLaplacian regularized low rank subspace clustering
The problem of fitting a union of subspaces to a collection of data points drawn from multiple subspaces is considered in this paper. In the traditional low rank representation model, the dictionary used to represent the data points is chosen as the data points themselves and thus the dictionary is corrupted with noise. This problem is solved in the low rank subspace clustering model which deco...
متن کامل